Multimodal Classification of Activities of Daily Living Inside Smart Homes
Identifieur interne : 000460 ( Main/Exploration ); précédent : 000459; suivant : 000461Multimodal Classification of Activities of Daily Living Inside Smart Homes
Auteurs : Vit Libal [États-Unis] ; Bhuvana Ramabhadran [États-Unis] ; Nadia Mana [Italie] ; Fabio Pianesi [Italie] ; Paul Chippendale [Italie] ; Oswald Lanz [Italie] ; Gerasimos Potamianos [Grèce]Source :
- Lecture Notes in Computer Science [ 0302-9743 ] ; 2009.
Abstract
Abstract: Smart homes for the aging population have recently started attracting the attention of the research community. One of the problems of interest is this of monitoring the activities of daily living (ADLs) of the elderly aiming at their protection and well-being. In this work, we present our initial efforts to automatically recognize ADLs using multimodal input from audio-visual sensors. For this purpose, and as part of Integrated Project Netcarity, far-field microphones and cameras have been installed inside an apartment and used to collect a corpus of ADLs, acted by multiple subjects. The resulting data streams are processed to generate perception-based acoustic features, as well as human location coordinates that are employed as visual features. The extracted features are then presented to Gaussian mixture models for their classification into a set of predefined ADLs. Our experimental results show that both acoustic and visual features are useful in ADL classification, but performance of the latter deteriorates when subject tracking becomes inaccurate. Furthermore, joint audio-visual classification by simple concatenative feature fusion significantly outperforms both unimodal classifiers.
Url:
DOI: 10.1007/978-3-642-02481-8_103
Affiliations:
Links toward previous steps (curation, corpus...)
- to stream Istex, to step Corpus: 001888
- to stream Istex, to step Curation: 001331
- to stream Istex, to step Checkpoint: 000164
- to stream Main, to step Merge: 000474
- to stream Main, to step Curation: 000460
Le document en format XML
<record><TEI wicri:istexFullTextTei="biblStruct"><teiHeader><fileDesc><titleStmt><title xml:lang="en">Multimodal Classification of Activities of Daily Living Inside Smart Homes</title><author><name sortKey="Libal, Vit" sort="Libal, Vit" uniqKey="Libal V" first="Vit" last="Libal">Vit Libal</name></author><author><name sortKey="Ramabhadran, Bhuvana" sort="Ramabhadran, Bhuvana" uniqKey="Ramabhadran B" first="Bhuvana" last="Ramabhadran">Bhuvana Ramabhadran</name></author><author><name sortKey="Mana, Nadia" sort="Mana, Nadia" uniqKey="Mana N" first="Nadia" last="Mana">Nadia Mana</name></author><author><name sortKey="Pianesi, Fabio" sort="Pianesi, Fabio" uniqKey="Pianesi F" first="Fabio" last="Pianesi">Fabio Pianesi</name></author><author><name sortKey="Chippendale, Paul" sort="Chippendale, Paul" uniqKey="Chippendale P" first="Paul" last="Chippendale">Paul Chippendale</name></author><author><name sortKey="Lanz, Oswald" sort="Lanz, Oswald" uniqKey="Lanz O" first="Oswald" last="Lanz">Oswald Lanz</name></author><author><name sortKey="Potamianos, Gerasimos" sort="Potamianos, Gerasimos" uniqKey="Potamianos G" first="Gerasimos" last="Potamianos">Gerasimos Potamianos</name></author></titleStmt><publicationStmt><idno type="wicri:source">ISTEX</idno><idno type="RBID">ISTEX:607462EB646ACEE6C81281696695B48A185AD4FE</idno><date when="2009" year="2009">2009</date><idno type="doi">10.1007/978-3-642-02481-8_103</idno><idno type="url">https://api.istex.fr/document/607462EB646ACEE6C81281696695B48A185AD4FE/fulltext/pdf</idno><idno type="wicri:Area/Istex/Corpus">001888</idno><idno type="wicri:explorRef" wicri:stream="Istex" wicri:step="Corpus" wicri:corpus="ISTEX">001888</idno><idno type="wicri:Area/Istex/Curation">001331</idno><idno type="wicri:Area/Istex/Checkpoint">000164</idno><idno type="wicri:explorRef" wicri:stream="Istex" wicri:step="Checkpoint">000164</idno><idno type="wicri:doubleKey">0302-9743:2009:Libal V:multimodal:classification:of</idno><idno type="wicri:Area/Main/Merge">000474</idno><idno type="wicri:Area/Main/Curation">000460</idno><idno type="wicri:Area/Main/Exploration">000460</idno></publicationStmt><sourceDesc><biblStruct><analytic><title level="a" type="main" xml:lang="en">Multimodal Classification of Activities of Daily Living Inside Smart Homes</title><author><name sortKey="Libal, Vit" sort="Libal, Vit" uniqKey="Libal V" first="Vit" last="Libal">Vit Libal</name><affiliation wicri:level="2"><country xml:lang="fr">États-Unis</country><wicri:regionArea>IBM Thomas J. Watson Research Center, Yorktown Heights, New York</wicri:regionArea><placeName><region type="state">État de New York</region></placeName></affiliation><affiliation></affiliation></author><author><name sortKey="Ramabhadran, Bhuvana" sort="Ramabhadran, Bhuvana" uniqKey="Ramabhadran B" first="Bhuvana" last="Ramabhadran">Bhuvana Ramabhadran</name><affiliation wicri:level="2"><country xml:lang="fr">États-Unis</country><wicri:regionArea>IBM Thomas J. Watson Research Center, Yorktown Heights, New York</wicri:regionArea><placeName><region type="state">État de New York</region></placeName></affiliation><affiliation></affiliation></author><author><name sortKey="Mana, Nadia" sort="Mana, Nadia" uniqKey="Mana N" first="Nadia" last="Mana">Nadia Mana</name><affiliation wicri:level="1"><country xml:lang="fr">Italie</country><wicri:regionArea>Fondazione Bruno Kessler (FBK), Trento</wicri:regionArea><wicri:noRegion>Trento</wicri:noRegion></affiliation><affiliation></affiliation></author><author><name sortKey="Pianesi, Fabio" sort="Pianesi, Fabio" uniqKey="Pianesi F" first="Fabio" last="Pianesi">Fabio Pianesi</name><affiliation wicri:level="1"><country xml:lang="fr">Italie</country><wicri:regionArea>Fondazione Bruno Kessler (FBK), Trento</wicri:regionArea><wicri:noRegion>Trento</wicri:noRegion></affiliation><affiliation></affiliation></author><author><name sortKey="Chippendale, Paul" sort="Chippendale, Paul" uniqKey="Chippendale P" first="Paul" last="Chippendale">Paul Chippendale</name><affiliation wicri:level="1"><country xml:lang="fr">Italie</country><wicri:regionArea>Fondazione Bruno Kessler (FBK), Trento</wicri:regionArea><wicri:noRegion>Trento</wicri:noRegion></affiliation><affiliation></affiliation></author><author><name sortKey="Lanz, Oswald" sort="Lanz, Oswald" uniqKey="Lanz O" first="Oswald" last="Lanz">Oswald Lanz</name><affiliation wicri:level="1"><country xml:lang="fr">Italie</country><wicri:regionArea>Fondazione Bruno Kessler (FBK), Trento</wicri:regionArea><wicri:noRegion>Trento</wicri:noRegion></affiliation><affiliation></affiliation></author><author><name sortKey="Potamianos, Gerasimos" sort="Potamianos, Gerasimos" uniqKey="Potamianos G" first="Gerasimos" last="Potamianos">Gerasimos Potamianos</name><affiliation wicri:level="1"><country xml:lang="fr">Grèce</country><wicri:regionArea>Institute of Computer Science (ICS), FORTH, Heraklion</wicri:regionArea><wicri:noRegion>Heraklion</wicri:noRegion></affiliation><affiliation></affiliation></author></analytic><monogr></monogr><series><title level="s">Lecture Notes in Computer Science</title><imprint><date>2009</date></imprint><idno type="ISSN">0302-9743</idno><idno type="eISSN">1611-3349</idno><idno type="ISSN">0302-9743</idno></series></biblStruct></sourceDesc><seriesStmt><idno type="ISSN">0302-9743</idno></seriesStmt></fileDesc><profileDesc><textClass></textClass><langUsage><language ident="en">en</language></langUsage></profileDesc></teiHeader><front><div type="abstract" xml:lang="en">Abstract: Smart homes for the aging population have recently started attracting the attention of the research community. One of the problems of interest is this of monitoring the activities of daily living (ADLs) of the elderly aiming at their protection and well-being. In this work, we present our initial efforts to automatically recognize ADLs using multimodal input from audio-visual sensors. For this purpose, and as part of Integrated Project Netcarity, far-field microphones and cameras have been installed inside an apartment and used to collect a corpus of ADLs, acted by multiple subjects. The resulting data streams are processed to generate perception-based acoustic features, as well as human location coordinates that are employed as visual features. The extracted features are then presented to Gaussian mixture models for their classification into a set of predefined ADLs. Our experimental results show that both acoustic and visual features are useful in ADL classification, but performance of the latter deteriorates when subject tracking becomes inaccurate. Furthermore, joint audio-visual classification by simple concatenative feature fusion significantly outperforms both unimodal classifiers.</div></front></TEI><affiliations><list><country><li>Grèce</li><li>Italie</li><li>États-Unis</li></country><region><li>État de New York</li></region></list><tree><country name="États-Unis"><region name="État de New York"><name sortKey="Libal, Vit" sort="Libal, Vit" uniqKey="Libal V" first="Vit" last="Libal">Vit Libal</name></region><name sortKey="Ramabhadran, Bhuvana" sort="Ramabhadran, Bhuvana" uniqKey="Ramabhadran B" first="Bhuvana" last="Ramabhadran">Bhuvana Ramabhadran</name></country><country name="Italie"><noRegion><name sortKey="Mana, Nadia" sort="Mana, Nadia" uniqKey="Mana N" first="Nadia" last="Mana">Nadia Mana</name></noRegion><name sortKey="Chippendale, Paul" sort="Chippendale, Paul" uniqKey="Chippendale P" first="Paul" last="Chippendale">Paul Chippendale</name><name sortKey="Lanz, Oswald" sort="Lanz, Oswald" uniqKey="Lanz O" first="Oswald" last="Lanz">Oswald Lanz</name><name sortKey="Pianesi, Fabio" sort="Pianesi, Fabio" uniqKey="Pianesi F" first="Fabio" last="Pianesi">Fabio Pianesi</name></country><country name="Grèce"><noRegion><name sortKey="Potamianos, Gerasimos" sort="Potamianos, Gerasimos" uniqKey="Potamianos G" first="Gerasimos" last="Potamianos">Gerasimos Potamianos</name></noRegion></country></tree></affiliations></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Wicri/Terre/explor/NickelMaghrebV1/Data/Main/Exploration
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000460 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd -nk 000460 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Wicri/Terre
|area= NickelMaghrebV1
|flux= Main
|étape= Exploration
|type= RBID
|clé= ISTEX:607462EB646ACEE6C81281696695B48A185AD4FE
|texte= Multimodal Classification of Activities of Daily Living Inside Smart Homes
}}
| This area was generated with Dilib version V0.6.27. |  |